1. Field of the Invention
The present invention relates generally to image processing systems, and in particular, to a method, apparatus, and article of manufacture for controlling and manipulating the warping or morphing of images.
2. Description of the Related Art
When processing and editing video images or frames (e.g., in the entertainment industry), a variety of technologies and tools may be used during editing and special effects processing. One type of technology is that of image metamorphosis (also referred to as morphing or warping). Image warping is the warping or distortion of a source image into a destination image. Such a distortion is enabled by applying 2D geometric transformations on the images to retain geometric alignment between features, while color interpolation blends their color. Image metamorphosis (or morphing) is the technique of slowly and smoothly deforming a source image from a front clip into a second destination image from a back clip (or vice versa). For example, a morphing operation may provide the ability to transform a young person into an elderly person, an animal into a person, or one object into another object. Thus, image morphing may utilize warping routines and operations to warp a first source image into the second destination image.
However, prior art morphing/warping applications fail to provide sufficient user control and manipulation capabilities to a user. Further, it may be difficult to perform a morphing/warping operation as images move in a scene. To more fully understand these disadvantages a more detailed description of morphing and warping may be useful.
Image morphing between a source and destination image may begin with establishing a correspondence between the images using primitives such as line segments, curves (splines), or points. Based on the correspondence, the spatial relationship between all of the points in both images are used to compute a mapping/warping function. Based on the mapping function, the two images may be said to be warped into alignment. The mapping may also be used to create intermediate/interpolated images that in turn enables the presentation (to a viewer/editor/user) of a smooth transition between the two images.
There are three important aspects in morphing: identifying feature sets, deriving a warp function, and deriving transition functions.
Identifying one or more features is important because it permits the user to identify those image landmarks that must remain in geometric alignment as the morph proceeds from the source image to the destination image. If features are not properly identified or matched properly, double-exposure artifacts common with cross dissolves (e.g., the fade-out of one image and the fade-in of another image) may be readily visible. For example, instead of a morphing operation, without selecting landmark features, the resulting video clip may appear merely as a cross-dissolve where the images do not appear to morph into each other. Examples of some common image landmarks that may be identified as a feature on a face may include a profile, eyes, nose, and mouth.
Features may be specified by identifying particular points, polylines, and curves. Feature/control points may then be identified from a specified feature by point sampling on the polyline or curve. The feature control points on the source image are then matched to corresponding feature control points on the destination image.
The warp function is the next important aspect in the morphing process. The warp function defines the smoothness or the interpolated/intermediate frames to be created between the source and destination images. If a warp function is rough, interpolated images may be distorted with discontinuities or abrupt deformations.
One prior art warp function is referred to as a free-form deformation (FFD). A FFD deforms an object by manipulating a lattice or grid containing the object. The manipulated grid determines a deformation function that specifies a new position for each point on the object. Thus, instead of manipulating the actual points on the object/image itself, the FFD allows the user to control the warp using control points (at the vertices of the grid) which affect the warp of the images.
The third aspect is that of transition control. Transition control defines the rate at which each pixel or points in the source image is transitioned or moved into the corresponding point on the destination image. By controlling the rate of transition for a particular feature, the morphing or warp of the image may appear more realistic or smooth. For example, the transition rate for the control points of a particular landmark feature (e.g., a nose) may be accelerated while other features may maintain a lower transition rate. Such a transition setting on a landmark feature may provide for a significant improvement in a morphing sequence and a more realistic appearing special effect.
In view of the above, the prior art identifies a three stage process for morphing/warping an image. First, features are specified in both a source image and a second image. These corresponding features are then matched to each other. The second step is to define the actual warp function that defines how each point in the source image transitions or moves into the corresponding point in the destination image. An FFD may be used to manipulate the points in this regard. The last step is defining the speed/rate at which the source point transitions to the destination point.
However, within the available prior art technologies, the user/video editor may lack the ability to interact intuitively with the editing software. Accordingly, what is needed is a video editing warping/morphing tool that allows the user to efficiently, easily, intuitively, and quickly control the morphing/warping process.